The present invention relates generally to the field of computer system design and in particular to a method and system for providing an internal machine hardware function, typically implemented in the main storage of a physical hardware-based computing system, in a virtual machine of a computing system. The invention provides an operational interface between a first computer system environment and a separate and secure storage environment, which is not available for the execution of customer programs. By implementing the operational interface, execution of a microcode-based hardware machine function, initially situated in the secured storage environment is provided to the user through one or more isolated virtual machines of the first computer system environment, so as to provide its functionality to programs running in other portions of the first computer system environment.
Many modern computer systems implement various types of clustered architectures. In particular, the Parallel Sysplex clustered system offered by International Business Machines Corporation (IBM) (Parallel Sysplex and IBM are registered trademarks of International Business Machines Corporation, the present assignee hereof) comprise one or more S/390 servers coupled via intersystem links to one or more Coupling Facilities (CFs) which provide a structured, shared electronic storage for the coupled servers (S/390 is a registered trademark of IBM).
Establishment of a Parallel Sysplex computing system can be a work-intensive task which involves the acquisition and customization of new hardware and software. It has therefore been found advantageous to offer customers the opportunity to determine the benefits derived from the Parallel Sysplex system implementation by providing an emulated Parallel Sysplex computing environment for customer testing.
The foregoing problem has been previously addressed by providing a facility whereby the function of the Parallel Sysplex system can be emulated within a single hardware machine. In U.S. Pat. No. 5,564,040 to Kubala (assigned to the present assignee and incorporated herein by reference), the program code for performing the functions of the Coupling Facility is loaded into a maximum of two logical partitions of main storage in the hardware machine. In the Kubala patent, the (server mode) CF memory partitions are non-user accessible and provide Coupling Facility functions by interacting with user-accessible (non-server mode) memory partitions to simulate the function of a Parallel Sysplex system. While the Kubala patent does provide a simulated Parallel Sysplex system, it is limited to implementations including only two simulated Coupling Facilities and requires the allocation of a logical partition of main storage for each of these Coupling Facility instances.
In accordance with the foregoing it is clear that a solution which provides a customer with the ability to simulate highly complex Parallel Sysplex implementations including up to eight Coupling Facilities in a single Parallel Sysplex system, or alternatively to test multiple Parallel Sysplex systems on a single hardware machine without requiring the actual hardware systems or storage necessary for the real hardware-based implementation would prove advantageous to such customers. Moreover, it would be further beneficial to provide this simulated hardware function within a software environment having pre-existing debugging facilities with which the customer could more-effectively test the virtualized hardware implementation.
In response to these identified user requirements, via the facilities of the present invention, the services of the Coupling Facility or any other microcode-based hardware machine function can now be provided in a virtual machine of VM/ESA (VM), which is referred to as a virtual machine hypervisor or supervisor of other operating systems (VM/ESA is a registered trademark of IBM). Consequently, customers may test and debug the programs that are to use Coupling Facility services in a simulated Parallel Sysplex system comprising virtual machines rather than real machines.
This virtualization, by utilizing pageable storage, also allows unlimited multiple copies of the function to exist within a single computing system, each in its own virtual machine. Accordingly, the VM operating system allows a user to emulate one or more complete, independent simulated Parallel Sysplex systems, each with one or more CFs, in a single machine or logical partition thereof. This enables customer testing of large complex multisystem configurations without requiring those customers to acquire the actual hardware required for the configurations, or to possess the larger real storage required for such testing.
An additional advantage of implementing the simulated hardware machine function within the VM operating system is the availability within VM of existing debugging facilities and the ease of virtual machine re-configuration therein of this virtual Parallel Sysplex implementation. Users of the VM operating system have long utilized these facilities to test other software-based features, and accordingly this familiarity serves to aid in the testing of the virtual Parallel Sysplex system.
Advantageous features offered via the present invention address the foregoing problems and shortcomings of the prior art, and in particular, include implementation of an operational interface for virtualizing an internal hardware-based machine capability of a computing system. The interface is implemented between the VM/ESA (VM) operating-system hypervisor, executing on the main processors of a machine, or Central Electronics Complex (CEC), and the Service Element (SE) of the CEC, a separate hardware and program-execution domain within the CEC that is not accessible to users of the system and is not available for the execution of user programs. This interface provides a facility whereby a programmed hardware-based machine function called the Coupling Facility (CF), which typically executes in the main storage of a single hardware machine or logical partition thereof, is established for execution in one or more isolated virtual machines of VM, where it can be used to provide service to programs in other virtual machines in the same hardware machine or logical partition through a VM simulation of a coupling-hardware interface.
In general, a hardware machine function may be implemented at the normal architected programming-interface level of the hardware machine, yet be an integral part of the machine definition and not distributed as a programming product. An example of such an implementation is the CF function, the virtualization of which forms the basis for the subsequent detailed description section of this patent. The program code providing this function resides in the Service Element (SE) of the hardware system.
The Coupling Facility is implemented in proprietary microcode termed Coupling Facility Control Code (CFCC), which is classified by IBM as Licensed Internal Code (LIC) and not intended for access by customers using the computer system. Accordingly, a further feature of the present invention creates a new class of xe2x80x9cencapsulatedxe2x80x9d virtual machines in VM to protect the proprietary nature of the implementation of the LIC function from unauthorized access in a virtual machine. This protection is additionally important since a violation of the integrity of the LIC function can have a negative effect on the operation of the multiple computing systems comprising a virtual Parallel Sysplex system, potentially creating difficult debugging situations.
In contrast to typical unrestricted virtual machines in VM which are available for logon and full use by a user, an encapsulated virtual machine permits no such logon and regular use. Moreover, the system commands which are normally available to virtual-machine users are restricted in an encapsulated virtual machine. Requests for the services of an encapsulated virtual machine, which may be issued by users or programs running on other virtual machines, are provided to the encapsulated virtual machines and are concurrently used to trigger testing of the encapsulated virtual machine environment to ensure the continued non-violation of the encapsulation.
The transformation of the hardware-based machine function (which in the preferred embodiment is the Coupling Facility function, but could include any type of machine function embodied in programmed code and typically executed by a hardware machine) to provide for its execution in a virtual machine is enabled through use of an address list created by VM and transferred, using machine facilities, from CEC main storage to the private storage of the SE. The address list is utilized by VM and the main processors of the CEC as Segment and Page Tables for use by Dynamic Address Translation (DAT) hardware during operation of a virtual machine. The same address list is used by the SE as a two-level list of real-main-storage locations, in virtual-address order, identifying the real-page-frame locations into which the machine-function code (i.e., the LIC) is to be transferred by the SE, which itself has no access to the DAT hardware.
The invention provides for the creation of a test environment on a single hardware machine in which VM can support one or more virtual Parallel Sysplex systems. Each virtual machine operates as a separate virtual Coupling Facility, allowing testing of large, complex Parallel Sysplex configurations without the need for all of the hardware comprising the eventual real Parallel Sysplex system. Advantageously, since the resultant virtual-machine Coupling Facilities are in pageable storage, the large cumulative amount of electronic storage required for the physical Parallel Sysplex implementation(s) is not required for the testing of the virtual Parallel Sysplex system.
The SE is a separate, unique computational environment within an S/390 CEC. The SE need not utilize an S/390 operating environment when performing its own functions. The SE contains its own processor, memory and direct-access storage devices (DASD) which operate asynchronously to the function of the CEC. The storage capacity of the SE is used to store program and data elements that are required by the SE to support the main computational engines of the CEC.
One of the services provided by the SE is the establishment, within a defined environment, of a machine function, such as the CFCC. Heretofore, these defined environments have been either a physically distinct CEC, or a logical partition of a CEC as in the case of the aforementioned Kubala patent. The establishment of the function is actuated via a request through an interface defined for such requests. The SE processor performs such requests on its own engine in response to requests for service. A hardware interface is provided for the transfer of data from the storage domain of the SE to the main storage of the computing system, where the transferred data is then accessible by the main processors of the system as data or programs to be executed.
In a preferred embodiment the present invention implements a transformation and loading process which is different from the processes used to establish the hardware machine function in its native environment. Previously, the user program would reside in one physical CEC or logical partition, and call for the services of the established CF function in another physical system (CEC) or a physically isolated logical partition of memory within the same CEC. In accordance with the present invention, the user program resides in a virtual machine, and calls another virtual machine including the established hardware-machine function for service through the VM-hypervisor intermediary. Either virtual machine may be a pageable guest of the VM operating system.
In its native environment, the machine function must occupy fixed real storage. In contrast, via the facilities of the present invention, the transformed function is pageable in its virtual-machine form. The virtualization of the machine function code in pageable storage provides VM the flexibility of emulating multiple large Parallel Sysplex systems within a single CEC or logical partition, without exceeding the physical storage of the computing system and without intersystem-coupling hardware.
In accordance with an embodiment of the present invention, VM in effect provides a virtual multiple-physical-system environment in a single physical system. Yet, in a very real sense, the same logical isolation exists among the interacting virtual machines (and, indeed, among the entire separate virtual Parallel Sysplex systems in the user""s test environment) as if they were in physically different machines.
In order to make the hardware machine function available in a virtual embodiment, the function is transformed from its stored form in the SE to a virtual-memory-addressable form in a virtual machine. Via a further feature of the invention, calls for the services of the transformed hardware machine function are interpreted by a new service layer provided for this purpose in VM. In this test scenario, such calls are made by the using programs running in their own virtual machines and are relayed through this VM service layer to the transformed machine function, residing in another virtual machine of the VM operating system.
In accordance with the present invention, the VM service layer emulates the native hardware-to-microcode interface of the machine function in the virtual-machine-to-virtual-machine communication, from the program requesting service in one virtual machine to the CFCC in another virtual machine which provides the requested service.
The transformation of the CF function from SE storage to the virtual-machine domain entails the transfer of the binary image of the CFCC from the private DASD of the SE to addressable virtual pages within the address space of one or more encapsulated virtual machines in which the virtual embodiment of the function is to reside.
The interface defined for any request to the SE is a Service Call (SERVC) instruction which causes the SE to be signalled by a main processor of the CEC. The instruction format provides fields that are used to specify the action, and parameters of that action, being requested of the SE.
In the present invention the SERVC instruction to establish a virtual copy of a hardware-machine function in a virtual machine specifies the function to be virtualized (i.e., the Coupling Facility function in a preferred embodiment) and the main storage locations in which the code providing the function must be placed. As a parameter of the SERVC, an address list is provided by VM to the SE.
In the SE environment, the address list is a two-level list of real main-storage locations in virtual-address order of the pages (4096-byte blocks) of an address space into which the CFCC is to be placed. Each entry in the list represents a page in the encapsulated virtual-machine(s) virtual address space. The SE transfers the bytes of the functional image of the CFCC in pages consecutively, from the beginning of the CFCC in the SE storage, to the real-page-frame locations in the main storage of the CEC, using the real addresses supplied as a part of each entry in the supplied address list. The interface includes a starting offset within the virtual address space. This offset can be used by VM to request the proper placement of the machine function within the virtual address space of the encapsulated virtual machine in which it will operate.
Since the machine function must be addressable within a virtual machine after its transformation from SE storage, the interface selected is modeled on the Dynamic Address Table (DAT) structure through which the main processors of the CEC access virtual-address locations, including the CFCC when it is in operation after being established in a virtual machine.
Accordingly, through the present invention, the aforementioned two-level address list is defined for the transformation of the machine function into the virtual-address environment. For operation of the virtual machine, the DAT tables must be in the main storage of the CEC or logical partition. For SE use, a copy of the specified DAT tables is transferred to the SE private storage for use in performing the transformation operation. From the standpoint of the SE, each entry in a first table of main storage addresses is an address of a second table that holds 1024 real-page-frame addresses. After pages of the CFCC have been transferred to main storage using the applicable addresses in the applicable second table, the SE advances to the next entry in the first table to find a new second table containing real-page-frame addresses for the next consecutive portion of the CFCC. To the SE, only the real addresses and the page-frame-validity bits in the table entries are meaningful, and any other portions of these tables are ignored.
In the VM environment, these tables are, precisely, the DAT Segment and Page Tables established and pre-allocated to define the virtual machine that is to contain the hardware-machine function. All virtual pages must be backed by real-storage frames during any request to the SE for virtualization of a machine function. After the transfer is complete, VM may migrate less-frequently-used pages of the function to its direct access storage device (DASD) page space in its normal storage-management operations. Completion of the SERVC instruction is signalled to VM by the SE by means of a hardware interruption. Facilities are provided for VM to check the completion status of the requested service call.
Via a further inventive feature herein a related SERVC function allows an operating system in the CEC to request the length of the microcode for any virtualizable hardware machine function so that the operating system may properly prepare for the transformation request by building a two-level table specifying enough virtual storage, and locking enough real-page frames, to contain the function.
In yet another inventive feature, a related SERVC function allows an operating system to request that one of the previously-described functions be halted before completion, should the need arise (as, for example, if a virtual machine receiving a transformed function is reset by a user control).